Statistical Significance of Ranking Paradoxes
نویسندگان
چکیده
When nonparametric statistical tests are used to rank-order a list of alternatives, Simpsonlike paradoxes arise, in which the individual parts give rise to a common decision, but the aggregate of those parts gives rise to a different decision. Haunsperger (Haunsperger 2003) and Bargagliotti (Bargagliotti, submitted manuscript, 2008) have shown that the KruskalWallis (Kruskal and Wallis 1952), Mann-Whitney (Mann and Whitney 1947), and Bhapkar’s V (Bhapkar 1961) nonparametric statistical tests are subject to these types of paradoxes. We further investigate these ranking paradoxes by showing that when they occur, the differences in ranking are not statistically significant.
منابع مشابه
Detecting and Sorting the Paradoxes Associated with Smartphone Use by Brazilian Professionals
The use of mobile wireless technologies brings associated paradoxes already identified in the literature. Thus, this research uses quantitative analysis to determine the perception of presence and intensity level of fourteen technological paradoxes related to the use of smartphone by professionals in Brazil. Besides, this study identifies the variables that impact the perception of each paradox...
متن کاملTesting University Rankings Statistically: Why this Perhaps is not such a Good Idea after All. Some Reflections on Statistical Power, Effect Size, Random Sampling and Imaginary Populations
In this paper we discuss and question the use of statistical significance tests in relation to university rankings as recently suggested. We outline the assumptions behind and interpretations of statistical significance tests and relate this to examples from the recent SCImago Institutions Ranking. By use of statistical power analyses and demonstration of effect sizes, we emphasize that importa...
متن کاملSignificance tests for the evaluation of ranking methods
This paper presents a statistical model that interprets the evaluation of ranking methods as a random experiment. This model predicts the variability of evaluation results, so that appropriate significance tests for the results can be derived. The paper concludes with an empirical validation of the model on a collocation extraction task.
متن کاملSimpson's Paradox, Lord's Paradox, and Suppression Effects are the same phenomenon – the reversal paradox
This article discusses three statistical paradoxes that pervade epidemiological research: Simpson's paradox, Lord's paradox, and suppression. These paradoxes have important implications for the interpretation of evidence from observational studies. This article uses hypothetical scenarios to illustrate how the three paradoxes are different manifestations of one phenomenon--the reversal paradox-...
متن کاملParadoxes in Learning and the Marginal Value of Information
W consider the Bayesian ranking and selection problem, in which one wishes to allocate an information collection budget as efficiently as possible to choose the best among several alternatives. In this problem, the marginal value of information is not concave, leading to algorithmic difficulties and apparent paradoxes. Among these paradoxes is that when there are many identical alternatives, it...
متن کامل